Restricting Supervised Learning: Feature Selection and Feature Space Partition

Many supervised learning problems are considered difficult to solve either because of the redundant features or because of the structural complexity of the generative function. Redundant features increase the learning noise and therefore decrease the prediction performance. Additionally, a number of problems in various applications such as bioinformatics or image processing, whose data are sampled in a high dimensional space, suffer the curse of dimensionality, and there are not enough observations to obtain good estimates. Therefore, it is necessary to reduce such features under consideration. Another issue of supervised learning is caused by the complexity of an unknown generative model. To obtain a low variance predictor, linear or other simple functions are normally suggested, but they usually result in high bias. Hence, a possible solution is to partition the feature space into multiple non-overlapping regions such that each region is simple enough to be classified easily. In this dissertation, we proposed several novel techniques for restricting supervised learning problems with respect to either feature selection or feature space partition. Among different feature selection methods, 1-norm regularization is advocated by many researchers because it incorporates feature selection as part of the learning process. We give special focus here on ranking problems because very little work has been done for ranking using L1 penalty. We present here a 1-norm support vector machine method to simultaneously find a linear ranking function and to perform feature subset selection in ranking problems. Additionally, because ranking is formulated as a classification task when pair-wise data are considered, it increases the computational complexity from linear to quadratic in terms of sample size. We also propose a convex hull reduction method to reduce this impact. The method was tested on one artificial data set and two benchmark real data sets, concrete compressive strength set and Abalone data set. Theoretically, by tuning the trade-off parameter between the 1-norm penalty and the empirical error, any desired size of feature subset could be achieved, but computing the whole solution path in terms of the trade-off parameter is extremely difficult. Therefore, using 1-norm regularization alone may not end up with a feature subset of small size. We propose a recursive feature selection method based on 1-norm regularization which can handle the multi-class setting effectively and efficiently. The selection is performed iteratively. In each iteration, a linear multi-class classifier is trained using 1-norm regularization, which leads to sparse weight vectors, i.e., many feature weights are exactly zero. Those zero-weight features are eliminated in the next iteration. The selection process has a fast rate of convergence. We tested our method on an earthworm microarray data set and the empirical results demonstrate that the selected features (genes) have very competitive discriminative power. Feature space partition separates a complex learning problem into multiple non-overlapping simple sub-problems. It is normally implemented in a hierarchical fashion. Different from decision tree, a leaf node of this hierarchical structure does not represent a single decision, but represents a region (sub-problem) that is solvable with respect to linear functions or other simple functions. In our work, we incorporate domain knowledge in the feature space partition process. We consider domain information encoded by discrete or categorical attributes. A discrete or categorical attribute provides a natural partition of the problem domain, and hence divides the original problem into several non-overlapping sub-problems. In this sense, the domain information is useful if the partition simplifies the learning task. However it is not trivial to select the discrete or categorical attribute that maximally simplify the learning task. A naive approach exhaustively searches all the possible restructured problems. It is computationally prohibitive when the number of discrete or categorical attributes is large. We describe a metric to rank attributes according to their potential to reduce the uncertainty of a classification task. It is quantified as a conditional entropy achieved using a set of optimal classifiers, each of which is built for a sub-problem defined by the attribute under consideration. To avoid high computational cost, we approximate the solution by the expected minimum conditional entropy with respect to random projections. This approach was tested on three artificial data sets, three cheminformatics data sets, and two leukemia gene expression data sets. Empirical results demonstrate that our method is capable of selecting a proper discrete or categorical attribute to simplify the problem, i.e., the performance of the classifier built for the restructured problem always beats that of the original problem. Restricting supervised learning is always about building simple learning functions using a limited number of features. Top Selected Pair (TSP) method builds simple classifiers based on very few (for example, two) features with simple arithmetic calculation. However, traditional TSP method only deals with static data. In this dissertation, we propose classification methods for time series data that only depend on a few pairs of features. Based on the different comparison strategies, we developed the following approaches: TSP based on average, TSP based on trend, and TSP based on trend and absolute difference amount. In addition, inspired by the idea of using two features, we propose a time series classification method based on few feature pairs using dynamic time warping and nearest neighbor

High-Order Residual Network for Light Field Super-Resolution

Plenoptic cameras usually sacrifice the spatial resolution of their SAIs to acquire geometry information from different viewpoints. Several methods have been proposed to mitigate such spatio-angular trade-off, but seldom make use of the structural properties of the light field (LF) data efficiently. In this paper, we propose a novel high-order residual network to learn the geometric features hierarchically from the LF for reconstruction. An important component in the proposed network is the high-order residual block (HRB), which learns the local geometric features by considering the information from all input views. After fully obtaining the local features learned from each HRB, our model extracts the representative geometric features for spatio-angular upsampling through the global residual learning. Additionally, a refinement network is followed to further enhance the spatial details by minimizing a perceptual loss. Compared with previous work, our model is tailored to the rich structure inherent in the LF, and therefore can reduce the artifacts near non-Lambertian and occlusion regions. Experimental results show that our approach enables high-quality reconstruction even in challenging regions and outperforms state-of-the-art single image or LF reconstruction methods with both quantitative measurements and visual evaluation.Comment: 9 pages, 14 figures, accepted by the thirty-fourth AAAI Conference on Artificial Intelligenc

Dynamic analysis of offset press gear-cylinder-bearing system applying finite element method

A dynamic model of offset press gear transmission system made up of gears, cylinders and bearings is proposed in this study. The model based on finite element method (FEM) includes some nonlinearity such as time-varying meshing stiffness, backlash, static transmission error and contact nonlinearity, which lead to complex nonlinear coupling. The Darren Bell principle and Lagrangian approach are applied to derive the motion equations of system, then the Newmark method is used to solve the equations for meshing force, acceleration, shoulder iron and rubber contact force. Eigenvalue solution is used to predict the critical speed, moreover, the influence of the radial and axial stiffness on the first-order critical speed is discussed. Considering the importance of acceleration and meshing force, the RMS value of acceleration and dynamic factor are also studied in this paper. The dynamic orbits of system are observed from the phase diagram, power spectrum, Lyapunov exponent and Poincare map. The figures clearly indicate that there are various forms of periodic and chaotic motions in different conditions. The simulation results show that with the increase of rotating speed, dynamic orbits transfer from periodic motion to chaotic motion in the cylinder discrete state

ContraGen: Effective Contrastive Learning For Causal Language Model

Despite exciting progress in large-scale language generation, the expressiveness of its representations is severely limited by the \textit{anisotropy} issue where the hidden representations are distributed into a narrow cone in the vector space. To address this issue, we present ContraGen, a novel contrastive learning framework to improve the representation with better uniformity and discrimination. We assess ContraGen on a wide range of downstream tasks in natural and programming languages. We show that ContraGen can effectively enhance both uniformity and discrimination of the representations and lead to the desired improvement on various language understanding tasks where discriminative representations are crucial for attaining good performance. Specifically, we attain $44\%$ relative improvement on the Semantic Textual Similarity tasks and $34\%$ on Code-to-Code Search tasks. Furthermore, by improving the expressiveness of the representations, ContraGen also boosts the source code generation capability with $9\%$ relative improvement on execution accuracy on the HumanEval benchmark.Comment: 10 page

Clinical-radiomics-based treatment decision support for KIT Exon 11 deletion in gastrointestinal stromal tumors: a multi-institutional retrospective study

Objectivegastrointestinal stromal tumors (GISTs) with KIT exon 11 deletions have more malignant clinical outcomes. A radiomics model was constructed for the preoperative prediction of KIT exon 11 deletion in GISTs.MethodsOverall, 126 patients with GISTs who underwent preoperative enhanced CT were included. GISTs were manually segmented using ITK-SNAP in the arterial phase (AP) and portal venous phase (PVP) images of enhanced CT. Features were extracted using Anaconda (version 4.2.0) with PyRadiomics. Radiomics models were constructed by LASSO. The clinical-radiomics model (combined model) was constructed by combining the clinical model with the best diagnostic effective radiomics model. ROC curves were used to compare the diagnostic effectiveness of radiomics model, clinical model, and combined model. Diagnostic effectiveness among radiomics model, clinical model and combine model were analyzed in external cohort (n=57). Statistics were carried out using R 3.6.1.ResultsThe Radscore showed favorable diagnostic efficacy. Among all radiomics models, the AP-PVP radiomics model exhibited excellent performance in the training cohort, with an AUC of 0.787 (95% CI: 0.687-0.866), which was verified in the test cohort (AUC=0.775, 95% CI: 0.608-0.895). Clinical features were also analyzed. Among the radiomics, clinical and combined models, the combined model showed favorable diagnostic efficacy in the training (AUC=0.863) and test cohorts (AUC=0.851). The combined model yielded the largest AUC of 0.829 (95% CI, 0.621–0.950) for the external validation of the combined model. GIST patients could be divided into high or low risk subgroups of recurrence and mortality by the Radscore.ConclusionThe radiomics models based on enhanced CT for predicting KIT exon 11 deletion mutations have good diagnostic performance

Hypidone Hydrochloride (YL-0919) Produces a Fast-Onset Reversal of the Behavioral and Synaptic Deficits Caused by Chronic Stress Exposure

Our previous study showed that hypidone hydrochloride (YL-0919), a partial serotonin 1A (5-HT1A) receptor agonist and 5-HT reuptake inhibitor, exerts a significant antidepressant effect in various animal models. The aim of the present study was to further investigate the underlying mechanisms and whether it could act as a fast-onset antidepressant. In the current study, depressive-like behavior was induced in rats by a chronic unpredictable stress (CUS) model and assessed with the Sucrose Preference Test (SPT). Treatment with YL-0919 (2.5 mg/kg, i.g.), but not with fluoxetine (Flx; 10 mg/kg, i.g.), caused a fast improvement in the SPT scores. In CUS-exposed rats, YL-0919 treatment for 5 days decreased the immobility time in a forced swimming test (FST), and a 10-day treatment decreased the latency to feed in a Novelty-Suppressed Feeding Test (NSFT). In addition to the behavioral tests, the effects of YL-0919 on synaptic protein expression were also evaluated. Western blotting showed that YL-0919 significantly enhanced the expression levels of synaptic proteins such as synapsin I, postsynaptic density protein 95 (PSD95), phosphorylated mammalian targeting of rapamycin (pmTOR) and brain-derived neurotrophic factor (BDNF) in the hippocampus. To determine how the mTOR signaling is involved in the fast-onset antidepressant-like effects of YL-0919, the mTOR-specific inhibitor rapamycin was administered intracerebroventricularly (i.c.v.) together with the YL-0919 treatment. The observed changes in behavioral tests and protein expression could be reversed by rapamycin treatment. This suggests that the fast-onset antidepressant effects of YL-0919 were partially caused by changes in synaptogenesis mediated by activation of mTOR pathways. Our data suggest that YL-0919 may be a powerful/effective antidepressant with fast-onset

DPHL: A DIA Pan-human Protein Mass Spectrometry Library for Robust Biomarker Discovery

To address the increasing need for detecting and validating protein biomarkers in clinical specimens, mass spectrometry (MS)-based targeted proteomic techniques, including the selected reaction monitoring (SRM), parallel reaction monitoring (PRM), and massively parallel data-independent acquisition (DIA), have been developed. For optimal performance, they require the fragment ion spectra of targeted peptides as prior knowledge. In this report, we describe a MS pipeline and spectral resource to support targeted proteomics studies for human tissue samples. To build the spectral resource, we integrated common open-source MS computational tools to assemble a freely accessible computational workflow based on Docker. We then applied the workflow to generate DPHL, a comprehensive DIA pan-human library, from 1096 data-dependent acquisition (DDA) MS raw files for 16 types of cancer samples. This extensive spectral resource was then applied to a proteomic study of 17 prostate cancer (PCa) patients. Thereafter, PRM validation was applied to a larger study of 57 PCa patients and the differential expression of three proteins in prostate tumor was validated. As a second application, the DPHL spectral resource was applied to a study consisting of plasma samples from 19 diffuse large B cell lymphoma (DLBCL) patients and 18 healthy control subjects. Differentially expressed proteins between DLBCL patients and healthy control subjects were detected by DIA-MS and confirmed by PRM. These data demonstrate that the DPHL supports DIA and PRM MS pipelines for robust protein biomarker discovery. DPHL is freely accessible at https://www.iprox.org/page/project.html?id=IPX0001400000

Potential of Core-Collapse Supernova Neutrino Detection at JUNO

JUNO is an underground neutrino observatory under construction in Jiangmen, China. It uses 20kton liquid scintillator as target, which enables it to detect supernova burst neutrinos of a large statistics for the next galactic core-collapse supernova (CCSN) and also pre-supernova neutrinos from the nearby CCSN progenitors. All flavors of supernova burst neutrinos can be detected by JUNO via several interaction channels, including inverse beta decay, elastic scattering on electron and proton, interactions on C12 nuclei, etc. This retains the possibility for JUNO to reconstruct the energy spectra of supernova burst neutrinos of all flavors. The real time monitoring systems based on FPGA and DAQ are under development in JUNO, which allow prompt alert and trigger-less data acquisition of CCSN events. The alert performances of both monitoring systems have been thoroughly studied using simulations. Moreover, once a CCSN is tagged, the system can give fast characterizations, such as directionality and light curve